DataStorm partial update demo #659
Conversation
pepone
requested review from
InsertCreativityHere,
bernardnormier,
Copilot and
externl
There was a problem hiding this comment.
Pull Request Overview
This PR adds a new DataStorm partial update demo that demonstrates how to use partial updates to efficiently publish changes to individual fields without sending the entire data structure.
Key changes:
- New demo showing writer publishing atmospheric condition updates (temperature and humidity)
- Reader subscribes to updates and uses registered updaters to process both full and partial updates
- Uses full updates when both fields change, partial updates when only one field changes
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| cpp/DataStorm/partialUpdate/Writer.cpp | Implements writer that publishes full and partial updates for atmospheric conditions |
| cpp/DataStorm/partialUpdate/Reader.cpp | Implements reader that subscribes to and displays atmospheric condition updates |
| cpp/DataStorm/partialUpdate/README.md | Documentation explaining the demo and how to build/run it |
| cpp/DataStorm/partialUpdate/CMakeLists.txt | CMake build configuration for the demo |
| cpp/DataStorm/partialUpdate/AtmosphericConditions.ice | Slice definition for data types and update tags |
| cpp/DataStorm/partialUpdate/.gitignore | Git ignore file for build artifacts |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
|
||
| module ClearSky | ||
| { | ||
|
|
| module ClearSky | ||
| { | ||
|
|
||
| enum UpdateTag |
There was a problem hiding this comment.
Would be nice to have doc comments on this enum.
| This demo illustrates the use of **partial updates** in DataStorm. | ||
|
|
||
| The demo uses Slice to define the `Demo::Stock` class in the `Stock.ice` file. | ||
| The writer publishes atmospheric conditions changes, when both temperature and humidity changes it publishes |
There was a problem hiding this comment.
| The writer publishes atmospheric conditions changes, when both temperature and humidity changes it publishes | |
| The writer publishes atmospheric conditions changes. When both temperature and humidity changes it publishes |
|
|
||
| The demo uses Slice to define the `Demo::Stock` class in the `Stock.ice` file. | ||
| The writer publishes atmospheric conditions changes, when both temperature and humidity changes it publishes | ||
| a full update, when only one field changes it publishes a partial update for the given field. |
There was a problem hiding this comment.
| a full update, when only one field changes it publishes a partial update for the given field. | |
| a full update, and when only one field changes it publishes a partial update for the given field. |
| ClearSky::UpdateTag::HumidityUpdated, | ||
| [](ClearSky::AtmosphericConditions& t, float humidity) { t.humidity = humidity; }); | ||
|
|
||
| // Create an any key reader, that read all the samples published on the topic. |
There was a problem hiding this comment.
| // Create an any key reader, that read all the samples published on the topic. | |
| // Create an any key reader, that reads all the samples published on the topic. |
| std::mt19937 gen{std::random_device{}()}; | ||
| std::uniform_int_distribution<> atmosphericDist{0, 10}; | ||
|
|
||
| ClearSky::AtmosphericConditions conditions{21.5f, 50.0f}; |
There was a problem hiding this comment.
We should comment these two lines as thy interact with datastorm. Are these some kind of initial conditions? This is a full update?
| [](ClearSky::AtmosphericConditions& t, float temperature) { t.temperature = temperature; }); | ||
|
|
||
| // A partial updater for the humidity field. | ||
| topic.setUpdater<float>( |
There was a problem hiding this comment.
what happens if we, for example, set this float to a string and make the lambda no-op, or maybe convert the string to a float. Does writer.partialUpdate throw?
| while (true) | ||
| { | ||
| // Generate random changes where either temperature or humidity changes about 50% of the time. | ||
| bool tempChanged = atmosphericDist(gen) > 5; |
There was a problem hiding this comment.
Why not just generate a random temperate/humidity fluctuations between something like -1.0 and +1.0.
Then all we need to check is that it's not too low or high.
There was a problem hiding this comment.
the point of changed is that we use partial updates when only one of the conditions change.
2 participants
This PR adds a new DataStorm demo that showcase partial update feature.
The DataStorm/stock demo already shows partial updates the idea is to replace DataStorm/stock with this simpler demo.